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SOME  COMPLEXITY  THEORY  FOR  CRYPTOGRAPHY 


INTRODUCTION 

The  cryptographic  strength  of  public  key  cryptosystems  usually  depends  on  the  underlying 
assumption  that  certain  known  mathematical  problems  are  difficult  to  solve.  For  example,  we  shall 
see  that  the  RSA  system  (named  after  its  inventors:  Rivest,  Shamir,  and  Adleman  [1])  is  a  cryptosys¬ 
tem  whose  “breaking”  can  depend  on  the  solution  of  a  “hard”  mathematical  problem,  i.e.,  the  fac¬ 
toring  problem.  Thus,  it  becomes  significant  for  cryptography  to  classify  in  some  way  those 
mathematical  problems  that  are  “hard.”  Complexity  theory  attempts  to  do  this.  This  report  presents 
a  short  introduction  to  complexity  theory.  The  motivation  for  this  study  is  its  usefulness  in  cryptogra¬ 
phy. 


This  report  also  presents  an  adequate  model,  or  simulator,  of  an  algorithm  or  effective 
procedure— the  Turing  machine.  We  also  discuss  a  universal  simulator  for  all  such  machines  (the 
theoretical  inspiration  for  the  stored-program  computer).  We  find  there  are  problems  for  which  no 
algorithmic  solution  can  ever  be  found.  Finally,  for  problems  which  have  solution  algorithms  we  dis¬ 
cuss  a  way  to  measure  the  relative  efficiency  of  these  algorithms. 

THE  RSA  SYSTEM 

Conventional  cryptologic  systems  have  the  disadvantage  that,  for  efficient  decoding,  a  key  to  the 
decoding  process  must  be  separately  and  securely  passed  to  the  receiver.  A  public  key  cryptosystem, 
on  the  other  hand  allows  any  participant  P  in  a  communications  network  to  publicize  his  encoding 
scheme  to  the  network.  Doing  so,  however,  does  not  disclose  the  key  to  the  decoding  process.  Any 
member,  S,  of  the  network  who  wishes  to  send  a  secret  communication  to  P  may  do  so  by  encoding  it 
with  P’s  known  coding  scheme,  and  only  P  will  be  able  to  decode  it.  Secure  communication  of  a 
decoding  key  is  not  necessary.  It  is  also  desirable  that  the  method  have  an  authentication  feature; 
i.e.,  S  can  encode  the  message  to  P  in  such  a  way  as  to  include  S’s  “signature.”  A  signature  is 
some  proof  that  this  particular  message  came  from  S. 

One  elegant  candidate  for  such  a  scheme  is  the  RSA  system.  To  describe  the  RSA  algorithm, 
we  need  some  notation  and  some  elementary  number  theory  [2]. 

Let  jV  =  { 1 ,  2 ,  3 ,  .  .  .  m ,  .  .  .  J  be  the  set  of  natural  numbers.  For  m  €  jV  let  <t>(m)  be  Euler’s 
^-function  of  m  where  <t>(m)  ~  the  number  of  natural  numbers,  k ,  such  that  k  <  m  and  the  greatest 
common  divisor  of  k  and  m  is  1  (denoted  here  by  greatest  common  denominator  (gcd)  {k<m)  =  1). 
For  a ,  b ,  any  integers  with  m ,  a  positive  integer  greater  than  l,  we  write  the  expression 

a  m  b(  mod  m ) , 

(read  “a  is  congruent  to  b  modulo  m”)  to  denote  the  fact  that  the  integer  m  exactly  divides  a  -  b . 
The  RSA  technique  makes  use  of  the  following  simple  number  theoretical  result: 

Euler's  Theorem:  If  a,  m  €  N  with  gcd  (a,m)  -  1,  then  a<t>(m)  s  1  mod  m. 

The  method  works  as  follows.  Two  large  prime  numbers  p  and  q  (about  100  digits  each)  are 
chosen  at  random,  and  p  q  =  n  is  computed.  Letting  m  =  <)>(n )  =  (p  -  \)(q  -  1).  a  large  random 
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number  y  is  chosen  such  that  gcd  (y,m )  =  1*  This  step  guarantees  the  existence  of  an  integer 
x ,  0  <  x  <  m ,  such  that 


x  •  v  ®  1  (  mod  m ) . 


Efficient  computer  algorithms  exist  for  producing  p,  q,  y,  and  jc.  The  message  to  be  encoded  is 
translated  into  a  string  of  integers  in  the  set  (0,  19  2,  .  .  .  v  26);  e.g.,  blank  =  0,  A  =  1, 
B  =  2,  .  .  .  Z  =26.  The  resulting  string  is  then  treated  as  a  single  number  7\  0  s  T  s  n  -  I,  or 
as  a  sequence  of  such  numbers.  The  enciphering  process,  finding  E*  (T),  consists  of  computing 

Ek(T)  m  P  (  mod  n) 

where  0  s  EK  (T)  <  n  -  1.  The  encryption  scheme  is  made  public  by  announcing  n  and  y.  Using 
common  terminology,  the  public  key  is 

AT,  =  |/i,y). 


To  decode  the  ciphertext  C  =  EK  (7),  one  computes 

Dk  (T)  =  Cx  S  Txy  (  mod  n). 


where  0  <  Cx  <  n  -  1 .  We  note  that 

Txy  =  s  7X  mod  n) 

by  Euler’s  theorem.  Since  T  <  ny  Cx  =  T  is  now  uniquely  determined.  The  key  to  the  decoding,  x, 
is  not  made  public.  Thus,  the  secret  decryption  key  is 

K2  —  x  . 


Both  the  enciphering  and  deciphering  processes  can  be  done  efficiently  by  computer.  Breaking 
the  algorithm,  however,  requires  finding  the  prime  factors  p  and  q  of  n  which,  at  present,  cannot  be 
done  efficiently.  If  n  is  a  200-digit  number  for  example,  it  is  estimated  that  finding  its  prime  factors, 
by  using  a  high-speed  computer  and  the  best  factoring  algorithms  currently  known,  could  require 
about  3  billion  years. 

Before  getting  involved  in  our  discussion  of  complexity  theory,  we  make  the  following  caveat. 
Although  some  mathematical  problem  may  prove  to  be  “hard,”  it  is  not  true  that  the  cryptosystem 
which  is  based  on  this  “hard”  problem  will  be  as  hard  to  break.  We  discuss  this  later  in  this  report. 

FINITE-STATE  MACHINES 

In  this  section,  we  consider  an  attempt  to  simulate  general  computation.  In  fact,  a  finite-state 
machine  simulates  computational  devices  such  as  modern  digital  computers.  We  see  how  adequate 
this  structure  is  as  a  model  of  computation  in  the  most  general  sense  by  characterizing  its  capabilities 
as  a  “recognizer.”  To  attempt  to  build  a  mathematical  model  describing  finite-state  machines,  we 
first  try  to  abstract  some  of  the  important  features. 

•  Operations  of  the  machine  are  synchronized .  We  only  look  at  the  machine  at  fixed  times,  or 

clock  pulses,  We  assume  the  machine  is  discrete  so  that  the  responses  to  an 

input  at  /,  appear  at  ti4rX. 

•  The  machine  is  deterministic ;  i.e.,  its  actions  in  response  to  a  given  sequence  of  inputs  are 
completely  predictable. 
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•  The  machine  responds  to  inputs . 

•  There  is  a  finite  number  of  states  the  machine  can  attain.  At  any  time  r,  the  machine  is  in 
exactly  one  of  these  states.  Which  state  it  will  be  in  at  ti+\  is  a  function  of  both  its  present 
state  and  present  input.  The  present  state,  however,  depends  upon  the  previous  state  and 
input  and  so  forth  back  to  the  initial  operation.  Thus,  the  state  of  the  machine  at  any  moment 
serves  as  a  form  of  memory  of  past  inputs. 

•  The  machine  is  capable  of  output.  The  nature  of  the  output  is  a  function  of  the  present  state 
of  the  machine.  Thus  it  also  depends  upon  past  inputs. 

A  modem  digital  computer  has  these  five  features.  Its  operations  are  synchronized  by  clock 
pulses  (although  very  rapid);  it  operates  in  a  deterministic  fashion  and  is  capable  of  responding  to 
inputs.  A  computer  is  composed  of  a  large  number  of  bistable  “on-off’  elements.  If  there  are  n 
such  elements,  there  are  altogether  2"  on-off  configurations  which  the  computer  can  be  in.  These 
configurations  are  the  states  of  the  computer,  and  this  number  is  finite  (although  very  large).  The 
present  state  of  the  computer  (the  present  memory  configuration)  reflects  its  history  of  past  states  and 
inputs.  Finally,  the  output  at  any  moment  depends  upon  the  present  state  of  the  machine. 

We  are  now  ready  for  the  formal  definition: 

Definition  1  ~  M  =  [5,  /,  O .  fs ,  fo]  is  a  finite-state  machine  if  5  is  a  finite  set  of  states,  /  is  a 
finite  set  of  input  symbols  (the  input  alphabet),  O  is  a  finite  set  of  output  symbols  (the  output  alpha¬ 
bet),  and  fs  and  f0  are  functions  where,  fs  :  S  x  /  —  S  and  fQ  :  S  —  O .  The  machine  is  always  ini¬ 
tialized  to  begin  in  a  fixed  starting  state,  called  s0  here. 

The  function  fs  is  the  next-state  function.  It  maps  a  (state,  input)  pair  to  a  state.  Thus,  the 
state  at  clock  pulse  +  state  (r,  +  ,),  is  obtained  as  follows: 

state  (/,.,)  =  /,(  state  (/,),  input  (r,)). 

The  function  fQ  is  the  output  function.  When  f0  is  applied  to  a  state  at  time  r,  ,  we  get 

output  (rf->  =  f0(  state  (r,)). 

Notice  that  the  effect  of  applying  function  f0  is  available  instantly,  but  the  effect  of  applying  fs  is  not 
available  until  the  next  clock  pulse.  To  describe  a  finite-state  machine,  we  can  use  either  of  two 
alternatives:  (a)  The  state  table  actually  lists  sets  5,  /,  and  O  and  tabulates  the  functions  fs  and  f(). 
(b)  The  state  graph ,  a  directed  graph,  has  each  state  of  M  with  its  corresponding  output  as  vertices, 
and  the  next-state  function  is  given  by  directed  edges  of  the  graph  with  each  edge  showing  the  input 
symbol(s)  that  produces  that  particular  state  change. 

To  illustrate  state  tables  and  graphs,  we  give  some  simple  examples. 

Example  I  —  Let  M  be  a  machine  with  S  =  |50,5,,52i,  7=0=  |0, 1),  and  fs  and  fo  defined 
by  the  following  state  table: 


Present  State 

Next  State 

Output 

Present  Input 

0  1 

So 

*1 

*0 

0 

s  I 

*2 

*1 

1 

*2 

*2 

50 

1 
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The  machine  M  begins  in  state  sa  which  has  an  output  of  0.  If  the  first  input  symbol  is  a  0,  the  next 
state  of  the  machine,  fs{s0, 0)  =  5,.  S,  has  an  output,  f0(s ,)  =  1.  If  the  next  input  symbol  is  1,  the 
machine  stays  in  state  s ,,  fs(s  j,  1)  =  with  output  1.  Continuing  this  procedure,  an  input  sequence 
01101  (read  left  to  right)  would  produce  the  following: 


Time 

to 

h 

WM 

wm 

wm 

m 

Input 

0 

1 

i 

0 

i 

— 

State 

50 

*1 

Si 

*2 

*50 

Output 

0 

1 

i 

ui  A 

l 

0 

The  initial  0  of  the  output  string  is  spurious— it  merely  reflects  the  starting  state,  not  the  result  of  any 
input.  The  state  graph  of  M  is  given  as  follows: 


Example  2  —  The  machine  Af  described  here  is  a  parity -check  machine.  When  the  input 
received  through  time  rt  contains  an  even  number  of  Is,  then  the  output  at  time  r/  +  l  is  1;  otherwise, 
the  output  is  0.  The  state  graph  of  M  is  given  as  follows: 


For  simplicity,  we  assume  that  our  machines  have  the  same  input  and  output  alphabet,  usually  / 
=  O  =  (0,1}.  Also,  we  denote  by  /*  and  O*  the  sets  of  all  strings  of  elements  of  /  and  0,  respec¬ 
tively  (here  we  include  the  empty  set,  <£).  Example  2  already  exhibits  a  finite-state  machine  acting  as 
a  “recognizer."  This  recognizer  signals  with  an  output  of  1  whenever  an  input  string  belonging  to  a 
particular  set  of  possible  input  strings  has  been  received.  The  machine  of  Example  2  recognizes  the 
set  of  all  strings  consisting  of  an  even  number  of  Is. 

Now,  we  want  to  see  precisely  which  sets  the  finite-state  machines  c.re  capable  of  recognizing. 
Recognition  is  possible  because  machine  states  can  have  a  limited  memory  of  past  inputs.  Even 
though  the  machine  is  Finite,  it  is  possible  for  a  particular  input  signal  to  affect  the  behavior  of  a 
machine  “forever."  However,  not  every  input  signal  can  do  this  and  there  are  some  classes  of  inputs 
that  require  remembering  so  much  information  that  no  machine  can  detect  them. 


To  avoid  writing  down  outputs,  we  designate  those  states  of  a  finite-state  machine  with  output  1 
as  final  states  and  use  a  double  circle  to  denote  then  in  the  state  graph.  Thus  we  give  the  following 
definition  of  recognition: 
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Definition  2  —  A  finite-state  machine  M  with  input  alphabet  /  recognizes  a  subset  5  of  /*  if  A/, 
beginning  in  state  s0  and  processing  an  input  string  a,  ends  in  a  final  state  if  and  only  if  a  €  S. 

We  next  introduce  compact  symbolism  to  describe  the  sets  of  interest  to  us.  We  describe  these 
sets  by  using  ‘‘regular  expressions”;  each  regular  expression  describes  a  particular  set.  First,  we 
define  what  regular  expressions  are;  then  we  see  how  a  regular  expression  describes  a  set*  We 
assume  here  that  I  is  some  finite  set  of  symbols;  later,  /  will  be  the  input  alphabet  for  a  finite-state 
machine. 

Definition  3  —  (Regular  expression  over  /) 

(a)  The  symbol  0  is  a  regular  expression;  the  symbol  X  (used  for  the  empty  string)  is  a  regular 
expression. 

(b)  The  symbol  i  for  any  /  €  /  is  a  regular  expression. 

(c)  If  A  and  B  are  regular  expressions,  then  (. AB ),  ( AVB ),  and  ( A )*  are  regular  expressions. 

Definition  4  —  (Regular  sets)  Any  set  represented  by  a  regular  expression  according  to  the  con¬ 
ventions  described  below  is  a  regular  set: 

0  represents  the  empty  set, 

X  represents  the  set  (X)  containing  the  empty  string, 

i  represents  the  set  {/). 

For  regular  expressions  A  and  B , 

(AB)  represent  the  set  of  all  elements  of  the  form  a(3  where  a  belongs  to  the  set  represented 

by  A  and  0  belongs  to  the  set  represented  by  B . 

(A\B)  represents  the  union  of  A ’s  set  and  B's  set. 

(A  )*  represents  the  concatenation  of  members  of  A  ’s  set. 

We  note  that  X,  the  empty  string,  is  a  member  of  the  set  represented  by  A*.  In  writing  regular 
expressions,  we  eliminate  parentheses  when  no  ambiguity  results.  We  will  also  be  a  little  sloppy  and 
say  things  like  “The  regular  set  0*  V  10”  instead  of  “The  set  represented  by  the  regular  expression 
0*  V  10.” 

Example  3  —  Here  we  give  some  regular  expressions  and  describe  the  set  each  one  represents. 


(a)  1*0(01)* 


(b)  0V1* 

(c)  (0V1)* 


(a')  Any  number  (including  none) 
of  Is,  followed  by  a  single  0,  fol¬ 
lowed  by  any  number  (including 
none)  of  01  pairs. 

(b')  A  single  0  or  any  number 
(including  none)  of  Is. 

(c')  Any  string  of  0s  and  Is, 
including  X. 
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(d)  1 1((10)*1 1)*  (00*)  ( d ')  A  nonempty  string  of  pairs 

of  Is  interspersed  with  any 
number  (including  none)  of  10 
pairs,  this  string  followed  by  at 
least  one  0. 

We  have  introduced  regular  sets  because,  as  we  will  see,  these  are  exactly  the  sets  finite-state 
machines  are  capable  of  recognizing.  Thus,  any  set  recognized  by  a  finite-state  machine  is  regular; 
and  conversely,  any  regular  set  can  be  recognized  by  a  finite-state  machine.  This  result  was  first 
proved  in  1956  by  Stephen  Kleene.  First,  we  show  that  any  set  recognized  by  a  finite-state  machine 
is  regular. 

We  represent  finite-state  machines  by  directed  graphs.  Temporarily,  we  enlarge  the  set  of 
machines  to  include  structures  whose  graphs  may  not  have  a  full  complement  of  arrows,  so  that  some 
states  under  a  given  input  symbol  may  have  no  next  state  defined.  If  we  call  such  structures 
Machines  (with  a  capital  A/),  then  a  (finite-state)  machine  is  a  special  case  of  a  Machine.  Although 
we  are  ultimately  interested  in  the  set  of  strings  taking  a  given  machine  from  its  starting  state  to  any 
final  state,  we  first  consider  the  set  of  strings  taking  a  Machine  from  any  one  state  to  another,  not 
necessarily  different,  state.  By  using  induction  on  the  size  of  the  Machine,  we  prove  that  such  a  set 
is  regular. 

For  the  base  step,  assume  we  have  a  Machine  with  only  one  state,  s0.  Let  K  =  {i|,/2,. . .  ,/*)  be 
the  set  of  input  symbols  for  which  the  next-state  function  on  s0  is  defined.  We  want  to  find  a  regular 
expression  for  the  set  of  all  strings  taking  M  from  s0  to  s0.  Since  there  is  nowhere  else  to  go,  any 
input  from  K*  does  this.  Thus,  the  regular  expression  is  (ij  v  /2v . . .  v  /*)*.  Note  that  the  set  includes 
X,  which  certainly  takes  M  from  s0  to  s0. 

Now,  we  assume  that  in  any  k -state  Machine,  the  set  of  strings  taking  the  Machine  from  any 
state  to  any  state  5„  is  regular.  Finally,  we  let  M  be  a  Machine  with  {K  +  1)  states,  and  we  let 
and  sn  be  states  in  Af.  We  consider  the  two  cases  sm  =  sn  and  sm  *  sn. 

For  the  case  5m  =  sn ,  we  first  consider  nonempty  strings  taking  M  from  back  to  sm  for  the 
first  time.  Such  strings  will  be  of  two  types: 

•  a  single  input  symbol  /  6  / ;  and 

•  a  string  of  the  form  ipotiq  where  ip>  iq  €/.  ip  moves  M  from  sm  to  a  different  state  sml.  a  is  a 
string  moving  M  from  sml  to  some,  not  necessarily  different,  state  but  keeping  it  away 
from  sm ;  and  then  iq  takes  M  from  s]m  back  to  sm . 

Let  A  be  the  set  of  all  input  strings  taking  M  from  sml  to  without  going  through  sm .  If  we 
delete  sm,  the  rest  of  the  Machine  is  a  tf-state  Machine,  and  A  is  regular  by  the  induction  hypothesis. 
For  a  fixed  ip  and  iq ,  ip  Aiq  is  a  regular  set.  The  set  B  of  all  strings  of  the  form  (2),  above,  is  the 
union  of  a  finite  number  of  such  sets  (taking  the  union  over  the  various  ips  and  iqs)\  hence,  B  is  reg¬ 
ular.  And  the  set  C  of  all  strings  previously  described  is  the  union  of  B  with  a  finite  number  of  sin¬ 
gle  input  symbols;  thus  C  is  also  regular.  Now  C*  denotes  the  set  of  concatenations  of  members  of 
C  and  describes  the  set  of  all  input  strings  taking  M  from  sm  to  sm\  C*  is  regular. 

Now  we  need  to  handle  the  second  case  where  *  sn .  Again,  we  first  consider  the  set  E  of  all 
strings  moving  M  from  sm  to  for  the  first  time.  Any  such  string  is  of  the  form  ai  where  a  takes  M 
from  to  some  5  lm  *  sn  but  keeps  it  away  from  sn ,  and  i  takes  M  from  5  lm  to  .  Let  D  be  the  set 
of  ail  input  strings  taking  M  from  sm  to  sIm  without  going  through  s„.  If  we  disconnect  sn ,  the  rest  of 
the  Machine  is  a  K- state  Machine,  and  so  D  is  regular  by  induction  hypothesis.  For  a  fixed  i ,  Di  is 
therefore  a  regular  set.  The  set  E  then  consists  of  the  union  of  a  finite  number  of  such  sets  (taking 
the  union  over  various  is);  E  is  also  regular.  Now  let  F  denote  the  set  of  all  strings  taking  M  from 
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sn  to  sn :  we  know  F  is  regular  by  the  previous  case.  The  regular  set  EF  is  then  the  set  of  all  input 
strings  taking  Af  from  sm  to  . 

We  have  shown  that  the  set  of  input  strings  taking  a  Machine  Af  from  any  one  state  to  any  one 
state  is  regular.  The  set  of  strings  taking  a  (finite-state)  machine  Af  from  s0  to  any  final  state  is  the 
union  of  a  finite  number  of  such  sets,  and  so  it  is  regular.  On  the  other  hand,  if  Af  has  no  final 

states,  the  empty  set  <f>  is  the  only  set  “recognized,”  and  <t>  is  also  regular. 

We  have  proved  the  first  half  of: 

Kleene’s  Theorem  ( Part  I) 

Any  set  recognized  by  a  finite-state  machine  is  regular. 

This  theorem  states  that  given  a  finite-state  machine  A/,  there  exists  a  regular  expression 
describing  the  set  of  strings  Af  recognizes.  The  proof,  however,  does  not  tell  us  how  to  find  such  an 
expression. 

The  other  half  of  the  Kleene  theorem  states  that  for  any  regular  set,  there  is  a  finite-state 

machine  recognizing  it.  To  prove  this  result,  we  will  introduce  a  new  kind  of  machine  called  a  non - 

deterministic  finite-state  machine.  This  machine  is  defined  as  an  ordinary  finite-state  machine  except 
that  for  each  state-input  pair,  the  next  state  need  not  be  uniquely  determined  and  there  is,  in  fact,  a 
set  of  possible  next  states;  this  set  could  even  be  <t>.  In  other  words,  the  state  function  fs  maps  5  x  / 
to  the  set  of  subsets  of  5 . 

Example  4  —  Here  is  the  state  table  and  state  graph  of  a  nondeterministic  machine  Af . 


Present  State 

Next  State 

Output 

Present 

0 

Input 

l 

so 

Bl 

S\ 

Si 

As  a  nondeterministic  machine  acts  upon  an  input  string  «,  the  first  input  symbol  processed 
leads  Af  from  the  starting  state  to  a  set  of  possible  next  states.  Each  of  these  states,  upon  processing 
the  second  symbol,  has  a  set  of  possible  next  states;  the  union  of  these  sets  is  the  set  of  possible  states 
for  Af  after  processing  two  symbols  of  a.  If  we  continue  this  procedure,  we  find  the  set  of  possible 
states  for  Af  after  processing  a.  If  any  of  the  states  in  this  set  is  a  final  state  of  Af ,  we  say  that  Af 
recognizes  a.  The  set  of  strings  so  recognized  is  the  set  recognized  by  Af . 

Example  4  ( continued )  —  The  nondeterministic  machine  of  Example  4  recognizes  the  set 
0*1(OV10*1)*.  For  any  string  a  in  this  set,  Af  has  a  possible  sequence  of  moves  that  would  result  in 
Af  being  in  a  final  state  at  the  end  of  processing  a. 
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A  nondeterministic  machine  M  does  not  operate  by  choosing  at  each  clock  pulse  some  next  state 
out  of  a  set  of  possible  next  states.  Rather,  it  operates  like  a  parallel  processor,  keeping  track  at  all 
times  of  all  its  possible  configurations.  ,  we  simulate  M  ’s  behavior  by  running  in  parallel  a  bunch  of 
deterministic  machines,  each  of  which  traces  out  a  different  possible  sequence  of  moves  for  M .  We 
can  also  simulate  A/'s  behavior  by  constructing  a  single  big  deterministic  machine  with  enough  states 
to  represent  all  of  A/’s  possible  configurations. 

Lemma  —  For  any  nondeterministic  machine  M  recognizing  a  set  S,  there  is  a  deterministic 
machine  Af'  also  recognizing  S. 

Proof  —  The  states  of  A/'  are  sets  of  states  of  A/.  If  s0  is  the  starting  state  of  A/,  then  (s0j  is  the 

starting  state  for  A/'.  For  each  state  \st . s,  |  of  A/'  and  each  input  symbol  / ,  we  find  the  next 

state  of  A/'  by  taking  the  union  of  the  set  of  next  states  for  s( ,  under  /  in  A/,  s,  under  i  in  A/,  etc.  A 
state  of  A/'  is  labeled  a  final  state  if  and  only  if  it  contains  a  final  state  of  A/. 


Example  4  (continued)  —  Here  we  give  the  deterministic  counterpart  of  the  nondeterministic 
machine  of  Example  4.  The  state  table  and  graph  follow: 


Present  State 

Next  State 

_ _ _ _ 

Output 

Present 

0 

Input 

1 

A  =  |s0| 

|S(,.S|| 

0 

B  =  |s  ,| 

|S|| 

till 

0 

C  =  |s:| 

|S|.S2| 

Uoi 

1 

D  =  |s  0.  s  |  ( 

fs0,s,| 

|S|,S2| 

0 

E  =  |s  ! ,  S  . | 

|S|.S2| 

|S0.S|| 

1 

For  example,  the  next  state  of  under  1  is  because  the  set  of  next  states  in  M  for 

under  1  is  \sx\  and  the  set  of  next  states  in  M  for  s2  under  !  is  |s0|.  From  the  state  graph  for  Af  \  we 
see  that  A/'  recognizes  0*I(0V10*1)*;  we  also  see  that  states  B  and  C  are  “unreachable’’  from  the 
starting  state  A  and  so  could  be  eliminated. 

In  our  example,  the  number  of  states  in  the  deterministic  machine  M'  is  close  to  the  number  of 
states  in  the  original  non-deterministic  machine  Af.  This  situation  may  not  be;  if  M  has  n  states.  A/' 
could  have  as  many  as  2"  -  1  states. 

Our  Lemma  says  that  we  gain  no  recognition  capabilities  by  considering  nondeterministic 
machines.  Therefore,  the  proof  of  Kleene’s  theorem  will  be  complete  if  we  show  that  for  any  regular 
set,  there  is  a  nondeterministic  finite-state  machine  recognizing  it.  We  prove  that  such  a  machine 
exists  by  showing  how  to  construct  it.  Because  the  definition  of  a  regular  expression  is  inductive,  we 
must  construct  our  machine  inductively.  We  let  /  be  the  set  of  symbols,  and  consider  various  types 
of  regular  expressions. 
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(1)  <t>  and  X.  A  trivial  machine  with  a  single,  nonfinal  state,  as  below  recognizes  </>. 


A  deterministic  machine  that  recognizes  X  is 


(2)  For  /  €  /  a  deterministic  machine  that  recognizes  i  is  /  -  (i| 


(Note  in  (1)  and  (2),  a  deterministic  machine  is  a  special  case  of  a  nondeterministic  machine.) 

We  now  assume  that  for  regular  expressions  A  and  there  are  nondeterministic  recognizers  MA 
and  Mb.  To  avoid  mixups,  we'll  also  assume  the  states  in  MA  and  the  states  in  MB  have  different 
names. 

(3)  AB  The  basic  idea  here  is  to  connect  the  two  machines  MA  and  MB  in  series  to  create  a 
machine  MAB  recognizing  AB .  The  set  of  states  for  MaB  is  the  union  of  the  sets  of  states  of  MA  and 
Mb  .  The  starting  state  for  MAB  is  the  starting  state  of  M A  and  the  final  states  of  MaB  are  the  final 
states  of  Mb.  Whenever  a  state-input  in  MA  could  take  MA  to  a  final  state,  we  want  to  allow  the  pos¬ 
sibility  of  jumping  instead  to  the  starting  state  of  MB ,  so  that  we  begin  to  process  strings  d  €  B  in  MB. 
Hence,  we  modify  the  state  table  for  MA  so  that  whenever  the  set  of  next  states  contains  a  final  state 
of  Ma  ,  we  add  the  starting  state  of  MR  to  the  set.  Then  for  any  ad  e  AB ,  there  is  a  sequence  of 
moves  taking  MAB  from  its  starting  state  through  the  actions  of  MA  on  a  and  to  the  pv^int  of  recogni¬ 
tion,  then  transferring  to  perform  the  actions  of  MB  on  d  until  d  is  recognized  by  MB:  hence,  ad  is 
recognized  by  MaB  . 

(4)  A\B:  The  basic  idea  here  is  to  connect  the  two  machines  MA  and  MB  in  parallel  to  create  a 
machine  MAWB  recognizing  A\B.  The  states  of  MAVB  are  the  states  of  MA  plus  the  states  of  MB 
plus  one  additional  state,  s,  designated  as  the  starting  state  for  MA\B.  The  final  states  of  MaWB  are 
the  final  states  of  MA  plus  the  final  states  of  MB.  When  we  process  the  first  symbol  /  of  a  string  7, 
we  want  to  allow  the  possibility  of  simulating  either  MA  s  actions  in  processing  /  beginning  in  its 
starting  state  sA  or  MB' s  actions  in  processing  /  beginning  in  its  starting  state  sB.  We  define  the  set 
of  next  states  for  s  under  i  to  be  the  union  of  the  set  of  next  states  sA  under  /  and  the  set  of  next 
states  of  sB  under  i .  Thus,  MAVB  processes  y  by  simulating  either  MA  or  MB%  recognizing  7  if  it  is 
recognized  by  either  MA  or  MB . 

(5)  A  *;  Ma  *  uses  the  set  of  states  of  MA  plus  an  additional  starting  state  T,  which  also  must  be 
a  final  state  in  order  to  recognize  X.  The  final  states  of  MA  are  also  final  states  of  MA  *.  If  /  is  the 
first  symbol  of  a  string  7.  then  MA  *  should  simulate  MA  s  actions  in  processing  /  beginning  in  its 
starting  state  sA  .  Thus  we  let  the  set  of  next  states  of  5  under  i  be  the  set  of  next  states  of  sA  under 
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i .  If  an  initial  segment  of  7  is  recognized  by  A  ,  we  need  to  be  able  to  reinitialize  at  once.  Hence  we 
modify  MA  so  that  the  set  of  next  states  for  any  final  state  and  input  j  contains  the  set  of  next  states 
for  s  under  j .  This  modification  allows  the  first  character  after  the  initial  segment  to  be  processed 
just  as  MA  would  do  it  starting  in  SA  .  Thus  MA  *  recognizes  A  *. 

We  note  that  slight  modifications  of  the  above  procedure  will  be  required  to  take  care  of  trouble¬ 
some  cases  involving  X.  To  construct  a  machine  for  1*0*.  for  example,  we  want  to  leave  the  starting 
state  for  the  machine  of  1*  as  a  final  state,  even  though  according  to  (3)  only  the  final  states  of  0* 
should  remain  final.  Similarly,  a  machine  for  1*0  would  call  for  a  transfer  on  0  from  the  starting 
state  of  the  machine  for  1*  to  the  final  state  of  the  machine  for  0. 

The  previous  procedure  should  be  viewed  as  a  canonical  procedure;  i.e.,  it  is  completely  general 
and  always  works.  But  for  any  particular  case,  we  may  be  able  to  come  up  with  a  much  simpler 
machine.  In  summary,  we  have  proven: 


Kleene's  Theorem  —  A  set  is  regular  if  and  only  if  it  is  recognized  by  some  finite-state  machine. 

This  theorem  outlines  the  limitations  as  well  as  the  capabilities  of  finite-state  machines,  as  there 
are  certainly  many  sets  that  are  not  regular;  e.g.,  5  -  (0”  ln  |n  >  0|  is  not  regular  where  an  stands 
for  a  string  of  n  copies  of  a .  (Notice  that  0*1*  does  not  do  the  job.)  By  Kleene's  theorem,  there  is 
no  finite-state  machine  capable  of  recognizing  S.  Yet  5  seems  like  such  a  4 ‘nice”  set,  and  surely  you 
or  I  could  count  a  string  of  0s  followed  by  Is  and  see  whether  we  had  the  same  number  of  0s  as  Is. 
This  lapse  seems  to  suggest  some  deficiency  in  our  use  of  a  finite-state  machine  as  a  model  of  a  com¬ 
putational  device.  We  will  try  to  remedy  this  in  the  next  section. 

TURING  MACHINES 

We  use  the  terms  “algorithm,”  “effective  procedure,”  and  “computational  procedure”  inter¬ 
changeably,  and  we  do  not  give  a  formal  definition  for  any  of  them.  Instead,  we  appeal  to  a  com¬ 
mon,  intuitive  understanding  of  an  algorithm  or  effective  procedure.  We  assume  that  any  input  to 
which  an  algorithm  is  to  be  applied  has  been  encoded  into  numeric  form,  usually  nonnegative 
integers,  just  as  input  for  an  actual  digital  computer  program  is  encoded  and  then  stored  in  binary 
form. 


Recalling  the  set  5  =  |0"  T  |  n  >0),  let  us  try  to  see  why  no  finite-state  machine  can  recognize 
it.  We  probably  consider  ourselves  to  be  finite-state  machines  and  imagine  that  our  brains,  being 
composed  of  a  large  number  of  cells,  can  only  take  on  a  finite,  although  immensely  large,  number  of 
configurations,  or  states.  We  feel,  however,  that  if  someone  presented  us  with  an  arbitrarily  long 
string  of  0s  followed  by  an  arbitrarily  long  string  of  Is,  we  could  detect  whether  the  number  of  0s 
and  Is  was  the  same. 

For  small  strings  of  0s  and  Is,  we  could  perhaps  just  look  at  the  strings  and  decide.  Thus  we 
can  tell  without  great  effort  that  000111  6  5  and  that  0001 W  5.  However,  for  the  string 
0000000000000001 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ,  we  must  devise  another  procedure;  e.g. ,  we  could  count  the 
number  of  0s  and  when  we  get  to  the  first  1 ,  we  write  that  number  down  (or  remember  it)  and  then 
we  begin  counting  the  Is.  (This  is  what  we  did  mentally  for  smaller  strings.)  But  we  have  now 
made  use  of  some  extra  memory  because  when  we  finish  counting  Is,  we  have  to  retrieve  the  number 
representing  the  number  of  0s  to  make  a  comparison.  But  such  “information  retrieval”  is  what  a 
finite-state  machine  cannot  do;  its  only  capacity  for  remembering  input  is  to  have  a  given  input  sym¬ 
bol  and  send  it  to  a  particular  state.  Suppose  we  attempt  to  build  a  finite-state  machine  to  recognize 
5.  We  could  count  the  number  of  0s  seen  by  having  each  new  0  move  us  to  a  new  state  of  the 
machine.  However,  since  the  number  of  states  of  any  given  machine  is  a  finite  number,  this  plan 
fails  if  the  number  of  0s  read  in  is  larger  than  this  finite  number,  so  our  machine  clearly  could  not 
process  0*  T  for  all  n .  In  fact,  if  we  think  of  solving  this  problem  on  an  actual  digital  computer,  we 
encounter  the  same  difficulty.  If  we  set  a  counter  as  we  read  in  the  0s,  we  might  get  an  overflow 
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because  our  counter  can  only  go  so  high.  To  process  O'1]"  for  arbitrarily  large  n  requires  that  we 
have  unlimited  auxiliary  memory  available  to  store  the  value  of  our  counter,  which  in  practice  cannot 
happen. 

Another  way  we  humans  might  consider  attacking  the  problem  of  recognizing  S  is  to  wait  until 
the  entire  string  has  been  presented  to  us.  We  would  then  go  to  one  end  of  the  string  and  cross  out  a 
0,  go  to  the  other  end  and  cross  out  a  1,  go  back  and  forth  to  cross  out  another  0-1  pair,  and  continue 
this  operation  until  we  run  out  of  Os  or  Is.  The  string  belongs  to  S  if  and  only  if  we  run  out  of  both 
at  the  same  time.  Although  this  approach  sounds  rather  different  from  the  first  one,  it  still  requires 
remembering  each  of  the  inputs  in  that  we  must  go  back  and  read  them  once  the  string  is  complete. 
A  finite-state  machine  cannot  reread  input. 

We  have  come  up  with  two  computational  procedures  to  decide,  given  a  string  of  Os  and  Is, 
whether  that  string  belongs  to  S .  Both  procedures  required  some  form  of  additional  memory  unavail¬ 
able  in  a  finite-state  machine.  Evidently,  the  finite-state  machine  is  not  a  model  of  the  most  general 
form  of  computational  procedure. 

To  simulate  more  general  computational  procedures  than  a  finite-state  machine  can  handle,  we 
use  a  Turing  machine  (invented  by  A.  M.  Turing  in  1936).  A  Turing  machine  is  essentially  a  finite- 
state  machine  with  the  added  ability  to  reread  its  input  and  also  to  erase  and  write  over  its  input,  and 
with  unlimited  auxiliary  memory— thus  overcoming  deficiencies  already  noted  about  finite-state 
machines. 

A  Turing  machine  consists  of  a  finite-state  machine  and  a  tape  divided  into  cells,  each  cell  con¬ 
taining,  at  most,  one  symbol  from  an  allowable  finite  alphabet.  At  any  one  instant,  only  a  finite 
number  of  cells  on  the  tape  are  nonblank.  We  use  the  special  symbol  b  to  denote  a  blank  cell.  The 
finite-state  unit,  through  its  read-write  head,  reads  one  cell  of  the  tape  at  any  given  moment  (see  fig¬ 
ure  below).  By  the  next  clock  pulse,  depending  upon  the  present  state  of  the  unit  and  the  symbol 
read,  the  unit  either  does  nothing  (halts)  or  completes  three  actions. 

1.  Print  a  symbol  from  the  alphabet  on  the  cell  read  (it  could  be  the  same  symbol  that’s  already 
there). 

2.  Go  to  the  next  state  (it  could  be  the  same  state  as  before). 

3.  Move  the  read- write  head  one  cell  left  or  right. 


b 

1 

1 

0 

0 

0 

1 

1 

1 

b 

b 

1 

Finite-state  Unit 

We  describe  the  action  of  any  particular  Turing  machine  by  a  set  of  quintuples  of  the  form 
(s,i,r,A  ,d)  where  a  and  i  indicate  the  present  state  and  the  tape  symbol  being  read  and  r  denotes 
the  symbol  printed,  s'  denotes  the  new  state,  and  d  denotes  the  direction  of  the  move  of  the  read- 
write  head  (/?  =  right,  L  =  left).  Thus  a  machine  in  the  configuration 
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b 

1 

1 

0 

1 

b 

I 


2 


if  acting  according  to  the  quintuple  (2, 1,0,  ltR)  would  move  to  the  configuration 


b 

b 

1 

0 

0 

1 

b 

[I 

1 

Definition  5  —  Let  S  be  a  finite  set  of  states  and  /  a  finite  set  of  tape  symbols  (the  tape  alpha¬ 
bet)  including  a  special  symbol  b.  A  Turing  machine  is  a  set  of  quintuples  of  the  form  ($,/,/',$  \d') 
where  s.s'  €5, i, /'.€/,  and  d$\RyL  \  and  where  no  two  quintuples  begin  with  the  same  s  and  i  sym¬ 
bols. 


The  restriction  that  no  two  quintuples  begin  with  the  same  s  and  i  symbols  ensures  that  the 
action  of  the  Turing  machine  is  deterministic  and  completely  specified  by  its  present  state  and  symbol 
read.  If  a  Turing  machine  gets  into  a  configuration  for  which  its  present  state  and  symbol  read  are 
not  the  first  two  symbols  of  any  quintuple,  the  machine  halts. 

Just  as  in  the  case  of  ordinary  finite-state  machines,  we  specify  a  fixed  starting  state,  denoted  by 
0,  in  which  the  machine  begins  any  computation.  We  also  assume  an  initial  configuration  for  the 
read- write  head,  namely,  that  it  is  positioned  over  the  farthest  left  nonblank  symbol  on  the  tape.  (If 
the  tape  is  initially  all  blank,  the  read-write  head  can  be  positioned  anywhere  to  start.) 

The  tape  serves  as  a  memory  for  a  Turing  machine,  and,  in  general,  the  machine  can  reread 
cells  of  the  tape.  It  can  also  write  on  the  tape;  therefore,  the  nonblank  portion  of  the  tape  can  be  as 
long  as  desired,  although  there  are  still  only  a  finite  number  of  nonblank  cells  at  any  time.  Hence  the 
machine  has  available  an  unbounded,  although  finite,  amount  of  storage.  The  limitations  of  finite- 
state  machines  observed  earlier  do  not  exist  for  Turing  machines,  so  Turing  machines  should  have 
considerably  higher  capabilities  than  finite-state  machines.  In  fact,  a  finite-state  machine  is  a  very 
special  case  of  a  Turing  machine,  one  that  always  prints  the  old  symbol  on  the  cell  read,  always 
moves  to  the  right  and  halts  on  the  symbol  b.  (A  Turing  machine  may  fail  to  halt,  e.g.,  by  endlessly 
cycling  or  by  moving  forever  along  the  tape.) 

Turing  machines  are  usually  used  to  do  two  kinds  of  jobs.  First,  they  can  be  used  as  recogniz¬ 
ers  and  second  to  compute  functions.  Here  we  only  discuss  their  role  as  recognizers,  much  as  we 
considered  finite-state  machines  as  recognizers.  We  give  a  similar  definition,  provided  we  first  define 
a  final  state  for  a  Turing  machine.  A  final  state  in  a  Turing  machine  is  one  that  is  not  the  first  sym¬ 
bol  in  any  quintuple.  Thus  upon  entering  a  final  state,  whatever  the  symbol  read,  the  Turing  machine 
halts. 


Definition  6  —  A  Turing  machine  T  with  tape  alphabet  /  recognizes  (accepts)  a  subset  5  of  /*  if 
7\  beginning  in  standard  initial  configuration  on  a  tape  containing  a  string  a  of  tape  symbols,  halts  in 
a  final  state  if  and  only  if  a  €  S . 
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Note  that  Definition  6  leaves  open  two  alternative  behaviors  for  T  when  applied  to  a  string  a  of 
tape  symbols  not  in  5.  T  may  halt  in  a  non-final  state  or  T  may  fail  to  halt  at  all. 

We  now  build  a  Turing  machine  to  recognize  5  =  (0/1  T  |  n  >  0|.  The  machine  is  based  on  our 

second  approach  of  sweeping  back  and  forth  crossing  out  0,  1  -  pairs. 

Example  5  —  Here  we  build  a  Turing  machine  that  will  recognize  S  =  (O'1  \n  |  n  >  0).  We  will 

use  one  additional  special  symbol,  call  it  X;  so  the  tape  alphabet  I  =  (0,  \tb,X\.  State  6  is  the  only 
final  state.  The  quintuples  making  up  T  follow  with  a  description  of  their  function. 


(0,b,b,6,R) 

Recognizes  the  empty  tape  (which  is  in  5). 

(0,  0,  X,  1,  R) 

Erases  the  left  most  0  and  begins  to  move 
right. 

(1,0,  0,  1  ,R) 

(1,  1,  1,  1,  R) 

(1,  X,  X,  2,  L) 
(1,  b,  b,  2,  L) 

Moves  right  in  state  1  until  it  reaches  the 
end  of  the  string;  then  moves  left  in  state  2. 

(2,  1,  X ,  3,  L) 

Erases  the  rightmost  1  and  begins  to  move 
left. 

(3,  1,  1,  3,  L) 

Moves  left  over  Is. 

(3,  0,  0,  4,  L) 

Goes  to  state  4  if  more  0s  are  left. 

(3,  X,  X,  5,  R) 

Goes  to  state  5  if  no  more  0s  in  string. 

(4,  0,  0,  4,  L) 

Moves  left  over  0s. 

(4,  X,  X,  0,  R) 

Finds  left  end  of  string  and  begins  sweep 
again. 

(5,  X,  X ,  6,  R) 

No  more  Is  in  string,  machine  accepts. 

Is  the  Turing  machine  a  better  model  of  an  effective  procedure  than  the  finite-state  machine? 
Although  our  concept  of  effective  procedure  is  an  intuitive  one,  we  are  quite  likely  to  agree  that  any 
procedure  computable  by  a  Turing  machine  is  an  effective  procedure  or  algorithm.  In  fact,  the  set  of 
quintuples  of  T  is  itself  the  algorithm;  as  a  finite  list  of  finite  instructions  that  can  be  carried  out 
mechanically,  it  satisfies  the  various  conditions  which  could  be  common  to  anyone’s  notion  of  an 
algorithm.  Given  the  simplicity  of  the  Turing  machine  definition,  it  is  startling,  however,  to  assert 
that  anything  computable  by  an  effective  procedure  can  also  be  processed  by  means  of  a  Turing 
machine.  This  is  the  statement  of  the: 

Church-Turing  Thesis  —  Any  process  which  could  naturally  be  called  an  effective  procedure  can 
be  realized  by  a  Turing  machine. 

Because  the  Church-Turing  thesis  equates  an  intuitive  idea  (effective  procedure  or  algorithm) 
with  a  mathematically  precise,  well-defined  idea  (the  Turing  machine),  it  can  never  be  formally 
proved  and  must  remain  a  “thesis,”  not  a  “theorem.”  What,  then  is  its  justification? 
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One  piece  of  evidence  is  that  whenever  a  procedure  everyone  could  agree  was  an  effective  pro¬ 
cedure  (according  to  his  or  her  own  insights  into  this  idea)  has  been  proposed  to  compute  something, 
someone  has  designed  a  Turing  machine  to  also  do  the  computation.  (Of  course  there  is  always  the 
nagging  thought  that  someday  this  might  not  happen.) 

Another  piece  of  evidence  is  that  other  mathematicians,  several  of  them  at  about  the  same  time 
that  Turing  developed  the  Turing  machine— late  1930s  or  early  1940s— also  proposed  models  of  effec¬ 
tive  procedures.  On  the  surface,  each  proposed  model  seems  unrelated  to  any  of  the  others.  Because 
all  of  these  models  are  formally  defined,  just  as  a  Turing  machine,  it  is  possible  to  consider  on  a  for¬ 
mal,  mathematical  basis  whether  any  of  them  are  equivalent.  These  models,  as  well  as  the  Turing 
machine,  have  been  proven  equivalent;  i.e.,  they  define  the  same  class  of  functions  which  suggests 
that  Turing  computability  embodies  everyone’s  concept  of  effective  procedure. 

The  Church-Turing  thesis  is  now  widely  accepted  as  a  working  tool  in  research  in  the  area  of 
complexity.  By  accepting  this  thesis,  we  have  accepted  the  Turing  machine  as  the  ultimate  model  of 
an  effective  computational  device.  Its  capabilities  exceed  those  of  any  actual  computer  that,  after  all, 
does  not  have  the  unlimited  tape  storage  of  a  Turing  machine.  It  is  remarkable  that  Turing  proposed 
this  concept  in  1936,  well  before  the  advent  of  the  modem  computer. 

COMPUTATIONAL  COMPLEXITY 

Suppose  we  have  an  algorithm  A  solving  problem  F.  Here  we  may  be  thinking  of  an  algorithm 
as  a  Turing  machine  or  as  an  actual  computer  program.  In  either  case,  we  are  interested  in  how  fast 
our  algorithm  works.  Is  it  possible  to  devise  an  algorithm  A'  to  solve  P  that  is  “faster”  (more  effi¬ 
cient)  than  algorithm  A,  if  the  number  of  A'  basic  operations  is  smaller  than  the  number  of  A  basic 
operations?  Of  course,  A'  and  A  must  have  comparable  basic  operations,  and  we  must  be  comparing 
A'  and  A  in  the  same  environment;  e.g.,  we  cannot  compare  the  number  of  steps  in  a  Turing  machine 
computation  with  the  number  of  steps  in  a  computation  done  in  some  higher  level  programming 
language.  We  use  Turing  machine  computations  as  our  environment;  by  the  Church-Turing  thesis, 
we  express  any  algorithm  as  a  Turing  machine  computation.  There  are  other  models  of  computation 
that  could  be  used,  such  as  a  radom  access  machine  (RAM).  A  RAM  works  somewhat  like  a  Turing 
machine,  but  its  allowable  operations  resemble  more  closely  those  in  actual  programming 
languages— there  are  arithmetic  operations,  branching  instructions,  etc. 

The  efficiency  of  an  algorithm  is  also  known  as  its  complexity  —  this  is  merely  some  sort  of 
measure  as  to  the  amount  of  work  the  algorithm  must  do.  A  straightforward  algorithm  in  its  logic 
may  still  require  a  large  number  of  steps  to  carry  out  on  a  Turing  machine,  for  example. 

Suppose  that  the  set  S  is  recognized  by  a  Turing  machine  T.  We  will  only  consider  cases  where 
T  halts  on  all  inputs  since  we  want  to  count  the  number  of  steps  in  7”s  computation  of  S,  and  we 
don’t  want  T  to  go  on  indefinitely.  As  T  does  a  computation,  we  encode  the  input  in  some  way  on 
7”s  tape,  start  T  in  standard  initial  configuration,  and  count  the  number  of  steps  (clock  pulses)  in  the 
computation  until  T  halts.  We  would  expect  that  the  number  of  steps  required  for  T  to  process  any 
a  €  S  (or  a  t  S)  would  be  a  function  of  the  length  of  the  input. 

Definition  7  —  Let  T  be  a  Turing  machine  that  halts  on  all  inputs.  If  the  maximum  number  of 
steps  for  a  computation  by  T  on  any  input  of  length  n  is  /(«),  then  T  is  of  time  complexity  t(n). 

Example  6  —  Consider  the  Turing  machine  of  Example  5  that  recognizes  the  set 
5  =  |0"  1"  |  a  a  0).  The  maximum  number  of  steps  in  a  computation  occurs  when  the  input  a$S. 
Suppose  an  input  of  length  n  is  a  member  of  S.  The  computation  first  moves  the  read-write  head 
beyond  the  right  end  of  the  input,  which  requires  n  steps.  The  read-write  head  then  sweeps  back  and 
forth  across  that  part  of  the  input  not  replaced  by  Xs.  This  process  requires  successively 
n,n  —  1,  n  -2,  ...,I  steps.  Recognition  requires  one  final  step.  Thus  the  total  number  of  steps  is 
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/I+(«+#l-l+...  +  l)+l 


n2  +  3n  +2 


This  Turing  machine  is  of  time  complexity  t(n)  =  (1/2)  ( n 2  4-3 n  +  2). 

Another  measure  of  efficiency,  which  we  will  not  consider  here,  is  the  space  complexity  of  a 
Turing  machine,  a  measure  of  the  amount  of  tape  the  machine  uses  as  a  function  of  input  length. 

Definition  8  —  Let  /  and  g  be  two  functions  from  N  —  N .  Then  /  and  g  are  of  the  same 
order  of  magnitude  if  there  exist  positive  constants  c  i  and  n  {  such  that  /  (n )  <  c  { g  (n )  for  all  n  >  n  , 
and  there  exist  positive  constants  c2  and  n2  such  that  g(n)  ^  c2f  (n)  for  all  n  ^  n2 

We  note  that  the  algorithm  of  Examples  5  and  6  is  of  order  n2.  Suppose  we  have  two  algo¬ 
rithms  to  do  the  same  job  and  their  time  complexities  are  of  different  orders  of  magnitude,  say  A  is  of 
order  n  and  A '  of  order  n2.  Even  if  each  step  in  a  computation  takes  only  0.0001  s,  this  difference 
will  affect  total  computation  time  as  n  grows  large.  Also  suppose  we  have  a  third  algorithm  A  " 
whose  time  complexity  is  an  exponential  function,  say  2" .  The  table  below  compares  total  computa¬ 
tion  time  for  A,  A\  and  A  "  under  various  input  lengths: 


Algorithm 

Order 

Size  of  Input 

10 

50 

100 

n 

0.001  s 

0.005  s 

0.01s 

n2 

0.01  s 

0.25  s 

Is 

KflHI 

2" 

0.1024  s 

3570  yr 

4  x  1016  centuries 

Because  of  the  immense  growth  rate  of  algorithms  not  of  polynomial  order,  these  are  not  useful 
for  large  values  of  n.  In  fact,  problems  for  which  no  polynomial  time  algorithms  exist  are  called 
intractable.  There  may,  however,  be  extenuating  circumstances.  When  an  algorithm  has  time  com¬ 
plexity  t{n)  =  2\  say,  at  least  one  input  of  length  n  requires  2"  steps,  but  the  average  case  may  run 
much  faster.  In  general,  however,  a  choice  between  possible  algorithms  for  a  given  problem,  or 
attempts  to  improve  a  given  algorithm,  should  concentrate  on  the  order  of  magnitude  of  the  time  com¬ 
plexity  functions  involved. 

Definition  9  —  P  is  the  collection  of  all  sets  recognizable  by  Turing  machine  of  polynomial  time 
complexity  (P  stands  for  polynomial  time). 

Consideration  of  set  recognition  in  Definition  9  is  not  as  restrictive  as  it  may  seem.  Since  a 
Turing  machine  for  which  a  time  complexity  can  be  determined  must  halt  on  all  inputs,  it  decides 
membership  in  a  set.  Furthermore,  many  problems  can  be  posed  as  set  decision  problems  through  a 
suitable  encoding  of  the  objects  involved  in  the  problem.  The  particular  encoding  scheme  we  use 
determines  the  length  of  the  input  string  for  a  given  instance  of  a  problem,  and  thus  may  affect  the 
time  complexity  of  an  algorithm  to  solve  the  problem.  However,  if  there  are  two  encodings  for  a 
given  problem,  such  that  inputs  under  each  encoding  can  be  transformed  in  polynomial  time  to 
corresponding  inputs  under  the  other  encoding,  then  if  one  encoding  results  in  a  set  belonging  to  P,  so 
does  the  other. 

The  situation  is  equally  pleasant  with  respect  to  alternative  models  of  algorithms  (Turing 
machine,  RAM’s  etc.).  A  problem  solvable  by  an  algorithm  with  polynomial  time  complexity  on  one 
model  is  solvable  by  an  algorithm  with  polynomial  time  on  another  model.  Thus  we  speak  of  a  prob¬ 
lem  belonging  to  P,  meaning  that  a  polynomial  time-bounded  algorithm  exists  for  its  solution,  without 
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having  to  specify  the  computational  device  that  carries  out  the  algorithm  or  the  details  of  the  encoding 
problem  for  that  device  (e.g.,  we  could  ask  does  the  Hamiltonian  circuit  problem  belong  to  P,  but  no 
one  yet  knows  the  answer  —  this  asks  if  an  arbitrary  graph  has  a  cycle  using  every  vertix  of  the 
graph.) 

The  decision  problem  for  Hamiltonian  circuits,  unlike  problems  such  as  the  Halting  problem  in 
Complexity  theory  (Does  an  algorithm  exist  to  decide,  given  a  Turing  machine  7  and  a  string  a, 
whether  7  begun  on  a  tape  containing  a  will  eventually  halt?)  and  the  word  problem  in  Group  theory 
(Does  an  algorithm  exist  to  decide,  given  the  generators  and  defining  relators— a  presentation— for  a 
group  and  a  word  from  the  group,  whether  the  word  can  be  transformed  to  the  identity?),  is  not 
unsolvable.  An  algorithm  exists  to  test  whether  an  arbitrary  graph  has  a  Hamiltonian  circuit,  viz,  the 
trial-and-error  approach  of  testing  all  possible  paths.  The  same  thing  is  true  of  the  factoring  problem. 
We  can  simulate  this  type  of  behavior  by  using  a  nondeterministic  Turing  machine  (NDTM).  An 
NDTM  is  defined  just  like  an  ordinary  Turing  machine  except  that  for  each  state-input  pair,  there  is  a 
set  of  applicable  quintuples  and  so,  possibly,  a  choice  for  the  Turing  machine's  behavior  at  that  point. 
(This  corresponds  to  the  situation,  e.g.,  with  the  Hamiltonian  circuit  problem  that  as  we  trace  out 
paths,  we  may  have  a  choice  of  possible  next  moves  every  time  we  come  to  a  vertix  of  the  graph.) 
Each  choice  (each  quintuple)  specifies  the  symbol  to  be  printed,  the  next  state,  and  the  direction  of 
motion  of  the  read-head.  We  think  of  the  NDTM  as  pursuing  all  of  its  possible  sequences  of  action 
in  parallel.  An  NDTM  T  recognizes ,  or  accepts ,  a  string  a  of  tape  symbols  if  7\  begun  in  standard 
configuration  on  a,  has  some  sequence  of  moves  leading  to  a  halt  in  a  final  state.  T  recognizes  the 
set  of  all  recognized  strings. 

Definition  10  —  Let  T  be  an  NDTM.  For  every  recognized  input  string  a  of  length  « ,  there  is 
at  least  one  sequence  of  moves  leading  to  a  final  state;  for  each  accepted  string,  consider  only  the 
shortest  sequence  of  moves  leading  to  acceptance.  If  the  maximum  number  of  steps  used  in  any  such 
sequence  accepting  a  string  of  length  n  is  t(n ),  then  T  is  of  time  complexity  t(n ). 

Definition  11  —  NP  is  the  collection  of  all  sets  recognizable  by  NDTMs  of  polynomial  time 
complexity.  (NP  stands  for  nondeterministic  polynomial  time.) 

Any  ordinary  (deterministic)  Turing  machine  is  a  trivial  NDTM,  so  it  is  clear  that  P  <  NP. 
Whether  P  is  a  proper  subset  of  NP  is  the  question  which  occupies  us  for  the  rest  of  this  section. 

As  in  the  corresponding  case  of  finite-state  machines  (see  our  Lemma  in  the  proof  of  Kleene’s 
theorem),  any  set  recognizable  by  an  NDTM  T  can  also  be  recognized  by  a  deterministic  Turing 
machine  7\  We  can  think  of  T'  acting  on  a  given  input  a  as  simulating  one  after  another  the  possi¬ 
ble  sequences  of  moves  T  could  make  on  a  until  a  is  accepted  or  all  possible  sequences  have  been 
tried  and  a  is  rejected.  Therefore,  although  nondeterminism  gains  us  no  new  capabilities,  we  would 
expect  it  to  gain  us  some  lower  time  complexity. 

Thus  if  the  time  complexity  for  T  is  /(*),  we  would  expect  the  time  complexity  t(n)  for  V  to  be 
higher  for  two  reasons.  T*  cannot  execute  sequences  of  moves  in  parallel  as  T  can;  it  must  do  them 
in  a  serial  fashion.  Also,  7'  gives  us  more  information  about  an  input  a  of  length  n ;  although  T  may 
give  an  answer  within  t(n)  units  of  time  only  if  a  is  accepted,  T '  always  gives  us  an  answer  (yes  or 
no)  about  any  input  a  within  t'(n )  units  of  time.  There  is  one  detail  we  glossed  over  in  discussing 
T'*s  simulation  of  T  on  a.  T  may  have  sequences  of  moves  that  do  not  halt;  if  7'  begins  simulating 
one  of  these  sequences,  how  does  it  know  when  to  give  up  and  try  another  sequence?  If  7  has  time 
complexity  tin),  then  7'  need  not  pursue  any  sequence  of  moves  for  longer  than  tin )  units  of  time.  If 
a  is  accepted  by  7,  there  is  some  sequence  that  will  do  the  job  within  this  time.  We  may  imagine 
7'’s  possible  actions  on  a  given  a  as  something  like  the  tree  shown  in  the  figure  below;  as  7'  simu¬ 
lates  7,  it  need  not  look  below  tin)  levels,  and  it  can  trace  out  each  branch  of  the  tree  that  far. 
Because  there  is  a  bound  b  on  the  maximum  number  of  possible  moves  7  can  make  at  any  point, 
there  are  at  most  b  branches  of  the  tree  at  any  vertix.  Thus  the  tree  can  have  at  most  b,{n)  separate 
paths,  each  of  length  at  most  r(n),  so  we  would  expect  some  exponential  expression  such  as 
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t{n)ban) 

to  be  the  time  complexity  for  T\  We  should  never  need  more  time  than  this,  but  probably  some 
input  of  length  n  for  some  n  might  require  this  much  time. 


The  previous  argument  seems  to  prove  that,  in  most  cases,  if  a  set  is  accepted  by  an  NDTM  of 
time  complexity  f(n),  it  will  probably  require  a  deterministic  machine  of  time  complexity  that  looks 
like  /(n  )bnn\  a  function  of  a  higher  order  of  magnitude.  Such  a  result  has  not  been  proven,  however. 
No  one  has  found  any  set  S  recognizable  by  an  NDTM  with  time  complexity  t(n )  for  which  no  deter¬ 
ministic  machine  of  complexity  t(n)  exists  to  recognize  S.  Although  there  are  certainly  sets  for  which 
such  a  deterministic  machine  has  not  been  found,  it  has  not  been  established  that  one  cannot  exist.  In 
particular,  whether  P  is  a  proper  subset  of  NP  is  an  open  question. 

There  are  many  famous  problems  such  as  the  Hamiltonian  circuit  problem,  the  factoring  prob¬ 
lem,  and  the  knapsack  problem  that  have  been  shown  to  be  in  NP,  i.e.,  they  are  representable  as  NP 
sets,  but  for  which  no  polynomial-bounded,  deterministic  solution  algorithm  has  been  found.  This 
fact  lends  weight  to  the  speculation  that  P  is  indeed  a  proper  subset  of  NP.  This  view  is  the  prevail¬ 
ing  one  in  complexity  theory  circles  today.  It  is  strengthened  by  work  begun  in  1971  on  a  class  of 
problems  known  as  NP-complete  (NPC)  problems.  Roughly,  if  a  problem  is  NPC,  it  is  NP  and  at 
least  as  hard  to  solve  as  any  other  NP  problem  in  that  if  it  could  be  shown  to  belong  to  P,  then  every 
NP  problem  would  belong  to  P  and  P  would  equal  NP. 

Many  problems  from  different  fields  (graph  theory,  number  theory,  etc.)  have  been  shown  to  be 
NPC".  For  example,  both  the  Hamiltonian  circuit  problem  and  the  knapsack  problem  are  NPC.  The 
NPC  problems  are  diverse,  and  the  search  for  efficient  (polynomially  bounded)  solution  procedures 
has  been  extensive.  In  view  of  the  so  far  unsuccessful  search  for  an  efficient  solution  procedure  for 
even  one  such  problem,  it  seems  likely  that  P  ^  NP.  On  the  practical  side,  however,  one  should  not 
look  too  long  for  a  quick  and  easy  algorithm  to  solve  any  NP  problem  one  may  encounter. 

CONCLUSION 

The  subject  of  complexity  theory  deals  with  the  following  two  aspects  of  any  problem:  the  most 
efficient  method  of  obtaining  a  solution  and  the  number  of  operations  needed  to  perform  this  task. 
The  idea  of  using  intractable  problems  in  the  design  of  cryptosystems  seems  to  be  attractive;  however, 
there  are  a  number  of  difficulties  with  this.  Shamir  [3)  has  pointed  out: 

(1)  Complexity  theory  deals  with  only  the  worst  possible  case  of  any  problem.  It  could  be  that 
only  very  few  instances  of  a  problem  are  truly  intractable.  A  cryptosystem  based  on  such  a  problem 
would  only  occasionally  be  secure. 

(2)  Complexity  theorists  assume  that  only  a  certain  amount  of  information  is  available  for  the 
solution  of  an  instance  of  a  problem.  Cryptoanalysts  frequently  have  much  more  information  at  their 
disposal,  such  as  corresponding  plaintext  and  ciphertext. 
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(3)  Given  any  particular  difficult  problem,  it  is  not  always  possible  to  convert  it  into  a  cryp¬ 
tosystem.  As  a  matter  of  fact,  most  known  public  key  techniques  are  based  on  only  two  NP- 
problems:  the  factoring  problem  and  the  knapsack  problem. 

Moreover  it  has  been  conjectured  that  the  breaking  of  any  public  key  cryptosystem  is  not  as 
hard  as  an  NPC  problem.  A  piece  of  evidence  which  supports  this  is  the  breaking  of  the  Merkle- 
Hellman  trapdoor  Knapsack  cryptosystem  [3],  despite  the  fact  that  the  knapsack  problem  is  itself 
NPC.  Furthermore,  it  would  be  very  desirable  to  have  a  proof  of  the  equivalence  of  the  problem  of 
breaking  the  RSA  system  and  the  factoring  problem.  At  this  time,  no  such  proof  exists.  Thus  the 
present  state  of  complexity  theory  is  inadequate  to  demonstrate  the  computational  infeasibility  of  any 
cryptosystem.  What  is  needed  are  new  measures  of  complexity  especially  tailored  to  the  problem  of 
cryptoanalysis.  Admittedly,  while  this  appears  to  be  a  very  difficult  mathematical  problem,  it  would 
be  worthwhile  pursuing  it  because  when  we  can  certify  the  security  of  cryptosystems  according  to 
such  measures  of  cryptocomplexity,  the  problem  of  secure  communications  will  be  solved. 
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